Evaluating the Evidence

Health outcomes research applies a wide range of methods to identify optimal approaches to determine optimal approaches to determine the effects of healthcare interventions and policies. Critical evaluation skills are necessary to navigate the evidence and identify studies that should influence clinical decision making and policy. A hierarchical “pyramid of evidence” that emphasizes randomized controlled clinical trials (RCTs) has been promulgated as the approach to judging study design and quality. Whereas this hierarchy may be suitable for the evaluation of efficacy, it may be inappropriate for many health outcomes research questions. This article examines the relevance of this construct to questions beyond those of therapeutic efficacy and discusses the essential role of study designs beyond RCTs. The strengths and weaknesses of study designs commonly encountered in the medical literature are reviewed. The article concludes with 2 case studies that apply the concepts reviewed and illustrate the need to match the study design with the research question. Of note, the concepts of internal validity, external validity, confounding, and bias are central to the discussion of the strengths and weaknesses of study designs in the medical literature. Readers not familiar with these concepts should refer to the Appendix in the onlineonly Data Supplement for clarification.